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ABSTRACT 

Objective: To identify barriers to completing and 
publishing pharmacy residency research projects 
from the perspective of program directors and 
former residents. 

Methods: This was a cross-sectional survey of 
pharmacy residency program directors and former 
post-graduate year one and two residents. Directors 
of pharmacy residency programs whose residents 
present their projects at the Western States 
Conference (n=216) were invited to complete an 
online survey and asked to forward the survey to 
former residents of their program in 2009, 2010, or 
201 1 . The survey focused on four broad areas: 1 ) 
demographic characteristics of the residency 
programs, directors, and residents; 2) perceived 
value of the research project; 3) perceived barriers 
with various stages of research; and 4) self- 
identified barriers to successful research project 
completion and publication. 
Results: A total of 32 program directors and 98 
residents completed the survey. The minority of 
programs offered formal residency research 
training. Both groups reported value in the research 
project as part of residency training. Significantly 
more directors reported obtaining institutional 
review board approval and working through the 
publication process as barriers to the research 
project (46.7% vs. 22.6% and 73.3% vs. 43.0%, 
respectively p<0.05) while residents were more 
likely to report collecting and analyzing the data as 
barriers (34.4% vs. 13.3% and 39.8% vs. 20.0%, 
respectively, p<0.05). Both groups self-identified 
time constraints and limitations in study design or 
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quality of the study as barriers. However, while 
program directors also indicated lack of resident 
motivation (65.5%), residents reported lack of 
mentorship or program structural issues (43.3%). 
Conclusion: Overall, while both groups found value 
in the residency research projects, there were 
barriers identified by both groups. The results of this 
study may provide areas of opportunity for 
improving the quality and publication rates of 
resident research projects. 

Keywords: Publishing; Research; Education, 
Pharmacy, Graduate; Internship, Nonmedical; 
Pharmacists; United States 

EL RETO DE PUBLICAR PROYECTOS DE 
INVESTIGACION DE LOS RESIDENTES EN 
FARMACIA DESDE LA PERSPECTIVA DE 
LOA DIRECTORES Y RESIDENTES DE 
PROGRAMAS DE RESIDENCIA 

RESUMEN 

Objetivo: Identificar las barreras para completar y 
publicar los proyectos de investigation de la 
residencia en farmacia desde la perspectiva de los 
directores de programas y de los antiguos 
residentes. 

Metodos: Este fue un estudio transversal de 
directores de programa de residencia y antiguos 
residentes post-graduados de ano 1 y 2. Se invito a 
completar un cuestionario online a los directores de 
programas de residencia cuyos residentes 
presentaron proyectos en la Western States 
Conference (n = 216) y se les pidio que pasasen el 
cuestionario a los antiguos residentes de sus 
programas de los anos 2009, 2010 o 201 1. El 
cuestionario se centraba en cuatro grandes areas: 1) 
caracteristicas demograficas de los programas de 
residencia, los directores y los residentes; 2) valor 
percibido del proyecto de investigation; 3) barreras 
percibidas en los diferentes pasos de la 
investigation; y 4) barreras auto-identificadas para 
la conclusion exitosa y publication del proyecto de 
investigation. 

Resultados: Un total de 32 directores de programas 
y 98 residentes completaron el cuestionario. Una 
minoria de programas ofrecia formation formal en 
investigation. Ambos grupos encontraron valor al 
proyecto de investigation, como parte de la 
formation de la residencia. Significativamente mas 
directores comunico que obtener la aprobacion de 
la comision de investigation de la institution y el 
trabajo de la publication eran barreras para el 
proyecto de investigation (46.7% vs. 22.6% y 
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73.3% vs. 43.0%, respectivamente p<0.05), 
mientras que los residentes reportaban con mas 
probabilidad como barreras la recogida de datos y 
el analisis como barreras (34.4% vs. 13.3% y 
39.8% vs. 20.0%, respectivamente, p<0.05). 
Ambos grupos auto-identifico la falta de tiempo y 
las limitaciones en la calidad del diseno del estudio 
o su calidad como barreras. Sin embargo, mientras 
que los directores tambien indicaban la falta de 
motivation de los residentes (65,5%), los residentes 
reportaron la falta de tutela o problemas 
estructurales del programa (43,3%). 
Conclusion: En general, mientras que los dos 
grupos encontraron valor en los proyectos de 
investigation en la residencia, habfa barreras 
identificadas por los dos grupos. Los resultados de 
este estudio pueden proporcionar areas de posible 
mejora de la calidad y las tasas de publication de 
los proyectos de investigation delos residentes. 

Palabras clave: Edition; Investigation; Education 
de Postgrado en Farmacia; Internado no Medico; 
Farmaceuticos; Estados Unidos 



INTRODUCTION 

The pharmacy profession is moving towards 
producing highly skilled clinicians with advanced 
clinical training. 1,2 Pharmacy residency programs 
play an important role in meeting this objective with 
the number of programs increasing in response to 
higher demand for more skilled clinicians. To 
receive accreditation by the American Society of 
Health-System Pharmacists (ASHP), pharmacy 
residency programs must provide opportunity for 
residents to "conduct a practice-related project 
using effective project management skills". 3 ' 4 
Projects can be original research, medication use 
evaluations (MUE), quality improvement projects, or 
problem solving exercises. 3 ' 4 Residents typically 
present their work at regional residency 
conferences and are often encouraged, although 
not typically required, to publish their findings in 
peer-reviewed journals. 

Recent studies demonstrate low publication rates of 
pharmacy resident projects. 5 " 7 Only 16% of resident 
projects presented at the Southeastern Residency 
Conference (SERC) were published. 5 Similarly, 
publication rates of projects presented at the 
Western States Conference (WSC) have been 
estimated to be between 4.3% and 6.3%. 6 7 

Pharmacy directors acknowledge that executing a 
successful research project is one of the most 
challenging aspects of residency training. 8 
However, the specific aspects of the research and 
publication process that are most challenging are 
not well defined. Identifying these aspects may 
prove useful for residency programs desiring to 
improve their research training and project 
publication rates. The purpose of this study was to 
identify barriers to pharmacy residency research 
project completion and publication from the 
perspective of the program directors and former 
residents. We hypothesized that perceived barriers 
to publishing resident research projects from the 



perspective of the residency director will differ from 
those of the resident. 

METHODS 

Study Design and Participants 

This was a cross-sectional survey of pharmacy 
residency program directors and former residents. 
Postgraduate year 1 (PGY1 ) and postgraduate year 
2 (PGY2) residents who would have presented 
projects at WSC in 2009, 2010, or 2011 were 
eligible to complete the survey. If a resident had 
completed both a PGY1 and PGY2, then they were 
asked to answer the survey based on their PGY2 
experience. This study was submitted to the 
institution review board and determined to be 
exempt from review. 

Survey Development and Content 

Two surveys specific to the perspectives of program 
directors and former residents, but which contained 
similar items for comparison purposes were 
developed. The surveys focused on four broad 
areas: 1) demographic characteristics of the 
residency programs, program directors, and former 
residents, 2) level of difficulty associated with 
various stages of research (rated on a Likert-scale: 
1=hardly challenging, 5=very challenging), 3) 
perceived value of the research project(rated on a 
Likert-scale: 1=strongly disagree, 5=strongly agree), 
and 4) self-identified barriers to successful research 
project completion (open-ended response). 

Survey Administration 

The ASHP residency program directory was used to 
identify residency directors of programs who would 
likely attend the WSC. The directors were sent an 
email containing a short explanation of the study 
and a link to the online survey (SurveyMonkey, Palo 
Alto, CA). Three additional emails were sent at 
seven to ten day intervals to non-responders to 
encourage completion of the survey. Program 
directors were asked to forward the email to their 
former residents who had completed a residency 
between 2009 and 2011. This timeframe was 
chosen as it allowed residents at least 12 months to 
submit their project for publication while limiting the 
potential for recall bias. 

Statistical Analysis 

Percentages were used to summarize categorical 
data. Likert-scale responses were dichotomized by 
grouping answers 1 through 3 and 4 and 5. 
Differences in categorical variables between 
program directors and former residents were 
compared using Pearson's chi-squared test of 
association or Fisher's exact test for small numbers. 
Responses to open-ended questions were 
categorized into major themes independently by two 
study investigators. Discrepancies in categorizing 
responses were resolved by consensus and 
involvement of a third investigator. Data analysis 
was performed using Microsoft Excel (Microsoft, 
Redmond, WA) or GraphPad (GraphPad Software, 
La Jolla, CA). A p-value of <0.05 was considered 
statistically significant. 
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Table 1. Residency program director, residency program, and resident characteristics 


Characteristic, % (n) 


Residency Program 
Director and Program 
(n=32) 


Resident 
(n=97) 


Years as a residency director 


< 5 years 
5 to 9 years 
> 1 0 years 


37.5 (12) 
37.5 (12) 
25.0 (8) 


N/A 
N/A 
N/A 


Types of residencies offered* 


General- PGY1 


87.5 (28) 


N/A 




Specialty - PGY2 
Community- PGY1 
Managed Care- PGY1 


40.6 (13) 
21.9 (7) 
9.4 (3) 


N/A 
N/A 
N/A 


Administration- PGY1/PGY2 


9.4 (3) 


N/A 


Years institution has offered residency 


< 2 years 
3 to 5 years 
6 to 1 0 years 
> 1 0 years 


3.1 (1) 
12.5 (4) 
21.9 (7) 
62.5 (20) 


N/A 
N/A 
N/A 
N/A 


Number of residents per year 


<2 
3 to 5 


43.8 (14) 
21.9 (7) 


N/A 
N/A 




6 to 15 
> 15 


31.3 (10) 
3.1 (1) 


N/A 
N/A 


Residency completed 


PGY1 
PGY2 


N/A 
N/A 


67.3 (66) 
31.6 (31) 


Number of publications 


None 
1 to 2 
3 to 4 
>5 


15.6 (5) 
37.5 (12) 
21.9 (7) 
25.0 (8) 


64.3 (63) 
31.6 (31) 
3.1 (3) 
1 (1) 


Comfort with the research process 


Very comfortable 
Comfortable 
Moderately comfortable 
Somewhat comfortable 
Not comfortable 


21.9 (7) 
28.1 (9) 
25.0 (8) 
25.0 (8) 
0.0 (0) 


6.1 (6) 

35.7 (35) 

37.8 (37) 
17.3 (17) 

2.0 (2) 


Status of residency project 

Writing in progress 
Submitted, not accepted, will not resubmit 


N/A 
N/A 


20.4 (20) 
1.0 (1) 


Submitted, not accepted, plan to resubmit 
Submitted, currently under revision 
Manuscript in press or published 
No plans to publish 


N/A 
N/A 
N/A 
N/A 


5.1 (5) 
10.2 (10) 
15.3(15) 
46.9 (46) 


Project types used to fulfill requirement* 

Original research 
Quality improvement 
Drug or medication use evaluation 
Problem-solving exercise 
Other 


81.3 (26) 
78.1 (25) 
65.6 (21) 
6.3 (2) 
3.8 (3) 


81.6 (80) 
42.9 (42) 
51.0 (50) 
z1 .4 (2.A ) 
2.0 (2) 


Program month that residency projects start 

1 st month 
2 nd or 3 rd month 
4 th or 5 th month 
> 6 months 


28.1 (9) 
59.4 (19) 
12.5 (4) 
0.0 (0) 


31.6 (31) 
57.1 (56) 
7.1 (7) 
3.1 (3) 


Formal research training offered 


41.9 (13) 


44.9 (44) 


Projects required to be in a format suitable for publication 


81.3 (26) 


83.7 (82) 


* Not mutually exclusive 

PGY1= postgraduate year 1; PGY2= postgraduate year 2 



RESULTS 

There were 32 pharmacy directors who completed 
the survey. The number of former residents who 
received the survey is unknown, however 97 
completed the survey. Demographic and program 
characteristics are detailed in Table 1 . The majority 
of program directors represented institutions 
offering PGY1 training (87.5%) and used a variety 
of projects types to fulfill the research requirement 
including: MUEs (65.6%), original research (81.3%), 
and quality improvement projects (78.1%). The 
minority of programs provided formal research 



training (41.9%) and most required projects to be in 
a format suitable for publication at the end of the 
residency (81.3%). In general, program directors 
were experienced (62.5% had been the program 
director for >5 years) and reported feeling 
comfortable or very comfortable (50.0%) with the 
research process. 

The majority of residents had completed only a 
PGY1 residency (67.3%). Consistent with the 
director responses, the minority of residents 
reported receiving formal research training (44.9%) 
during their residency and that their project was 
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Table 2. Perceived value of pharmacy resident research projects 


Survey Item, median (%, n) 


Residency 
Program Director 

m— JUJ 


Resident 
(n=93) 


P-value 


The results of my research or practice-related project 
were valuable to my institution 


4.5 

(73.3%, 22) 


4.0 
(64.5%, 60) 


0.50 


The research or practice-related project adequately 
prepared me to conduct research in the future 


3.0 
(46.7%, 14) 


3.5 
(50.5%, 47) 


0.87 


The research or practice-related project provided me 
adequate management/administrative skills 


4.0 

(60.0%, 18) 


4.0 
(59.1%, 55) 


0.93 


The research or practice-related project provided 
expanded career opportunities 


3.0 
(33.3%, 10) 


3.0 
(35.5%, 33) 


0.97 


Rated using a 5-point Likert scale, where 1=strongly disagree and 5=strong agree. Responses dichotomized and 
presented by proportion who reported strongly agree or agree as well as median values. 



required to be in a format suitable for publication by 
the end of the residency (83.7%). Almost half of 
resident respondents (46.9%) were not planning to 
publish their projects while 15.3% reported their 
project was either published or in press. 

Overall, both the program directors and residents 
felt the projects were beneficial to the institution 
(73.3% vs. 64.5%, respectively, p=0.50) and 
provided an opportunity to develop adequate 
management and administrative skills (60.0% vs. 
59.1%, respectively, p=0.93) (Table 2). 
Approximately half of program director and resident 
respondents felt resident projects adequately 
prepared them for conducting future research 
(46.7% vs. 50.5%, respectively, p=0.87). Only about 
a third of program director and resident respondents 
thought that conducting a resident project expanded 
career opportunities (33.3% vs. 35.5%, respectively, 
p=0.97). 

The most commonly reported barriers to completing 
resident projects reported by directors and residents 
were developing a realistic timeline to complete the 
project in one year (40% vs. 45.2%, respectively, 
p=0.67) and working through the publication 
process (73.3% vs. 43.0%, respectively, p<0.01) 
(Table 3). However, 46.7% of program directors 
identified obtaining IRB or department approval as a 
significant barrier compared to only 22.6% of 
residents (p=0.02). There was also a significant 



difference between groups in the perceived barrier 
posed by both collecting (13.3% vs. 34.4%, p=0.02) 
and analyzing data (20% vs. 39.8%, p=0.04) with 
residents viewing these aspects as more 
challenging than the directors. 

There were 29 program directors and 90 residents 
who identified at least one barrier in the open 
response portion of the survey (Table 4). Major 
themes that emerged included: time constraints, 
lack of resident motivation, IRB approval or 
compliance with other rules or regulations, 
limitations in study design or quality of the study, 
effective mentorship or program structural issues, 
resident knowledge gaps, and continuity issues 
following residency completion. Both program 
directors and residents reported that two of the top 
three barriers to publication were time constraints 
(58.6% vs. 62.2%, respectively) and limitations with 
the study design or quality of the study (59.6% vs. 
56.6%, respectively). However, as the third barrier, 
program directors cited the theme of resident 
motivation (65.5%) while residents cited the theme 
of effective mentorship or issues related to the 
program structure (43.3%). 

DISCUSSION 

While some literature currently exists assessing the 
perceived value of residency research projects 9 , 
limited information exists identifying barriers to 



Table 3. Comparison of perceived barriers to completing residency research projects* 


Research Area, median, (%, n) 


Residency 
Program 
Director 
(n=30) 


Resident 
(n=93) 


P-value 


Identify a mentor to guide the resident through research process 


2.0 

(16.7%, 5) 


2.0 
(10.8%, 10) 


0.52 


Identify an original research question 


3.0 

(33.3%, 10) 


3.0 
(31.2%, 29) 


0.82 


Formulate an appropriate study design and research protocol 


3.0 

(26.7%, 8) 


3.0 
(25.8%, 24) 


0.93 


Develop a realistic timeline to complete the project in one year 


3.0 
(40.0%, 12) 


3.0 
(45.2%, 42) 


0.67 


Obtain IRB and/or department approval 


3.0 

(46.7%, 14) 


2.0 
(22.6%, 21) 


0.02 


Collect data 


2.0 

(13.3%, 4) 


3.0 
(34.4%, 32) 


0.02 


Analyze data 


2.0 

(20.0%, 6) 


3.0 
(39.8%, 37) 


0.04 


Working through the publication process 


5.0 

(73.3%, 22) 


4.0 
(43.0%, 40) 


<0.01 


* Not mutually exclusive 
IRB, institutional review board 

Rated using a 5-point Likert scale, where 1=hardly challenging and 5=very challenging. Responses dichotomized and 
presented as proportion who reported very challenging or challenging. 
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Table 4. Self-identified barriers to completing and publishing residency projects* 


Major Theme, % (n) 


Residency 
Program Director 
(n-29) 


Resident 
(n=90) 


Time 


58.6 

/ -1 "7\ 

(1 /) 


62.2 
(5b) 


Resident motivation 


65.5 
(19) 


27.8 
(25) 


Limitations in study design or quality of the study 


58.6 
(1 /) 


56.7 

/EM \ 

(51) 


Mentorship or program structural issues 


24.1 
(7) 


43.3 
(39) 


Institutional Review Board and other rules and regulations 


17.2 

(5) 


8.9 
(8) 


Resident knowledge gap 


17.2 

(5) 


15.6 
(25) 


Continuity post-residency 


10.3 
(3) 


3.3 
(3) 


* Not mutually exclusive 



conducting and publishing these projects. The 
barriers identified in our work are similar to those 
identified with medical resident research training 
and practicing pharmacists. 10,11 We hypothesized 
that perceived challenges may be different between 
residency program directors and former residents 
and indeed, we learned that while there were many 
similarities between the groups, there were 
important differences in key areas. 

Both program directors and residents valued the 
pharmacy residency projects, felt they were 
beneficial to their institution, and helped residents 
develop project management and administrative 
skills. These findings support those of another study 
aimed at documenting the perceived value of 
resident projects. 9 It is our belief that an important 
component of the ASHP resident project 
requirement is to increase the resident's 
competence and comfort with the research process 
so that they will participate in practice-based 
research in some capacity throughout their careers. 
While our study did not evaluate research 
competence, only about half of directors and 
residents reported feeling comfortable with the 
research process. Similarly, about half of 
respondents agreed that the residency project 
adequately prepared them to conduct future 
research, also supporting similar finding from an 
earlier study. 12,13 

Not surprisingly, common barriers identified by both 
program directors and residents involved generating 
quality projects that could be successfully 
completed within the time constraints of a one year 
residency. For a variety of reasons, this likely 
translates to low publication rates. Issues 
surrounding time constraints emerged as a 
recurring theme at all points of the research process 
including obtaining IRB approval, data collection 
and analysis, and manuscript preparation. 
Residents also noted that competing residency 
responsibilities further diminished their ability to 
work on research throughout the year. Time 
constraints may also result in many programs opting 
to engage residents in institution-specific small 
scale retrospective studies, such as MUEs or quality 
assurance projects, which often have limited 
external validity and are more difficult to publish. 
Most respondents indicated that projects were 



initiated between the second and third residency 
month. Recognizing that the first month of residency 
is extremely busy for both the program and resident, 
beginning projects as soon as possible during this 
time may help alleviate time constraints. 

As discussed previously, while program directors 
and residents agreed on the first two self-identified 
barriers to project publication, they differed on the 
third. Whereas program directors frequently cited a 
theme relating to resident motivation, residents 
identified a theme relating to effective mentorship 
and program support. While reasons underlying this 
disconnect are not fully addressed by this study, we 
believe that our study results provide some insight 
and potential areas of opportunity. First, more than 
half of both program director and resident 
respondents reported a lack of a formal research 
training process and individuals from both groups 
identified resident knowledge gaps as a top barrier 
to publication. Studies have shown that objectively 
assessed research-related knowledge of residents 
is low after residency completion. 13,14 As a result, 
approaching the research requirement from the 
same structured process as many of the other 
residency components may help increase resident 
confidence and competence in the research 
process. Recognizing that this may not be feasible 
at all institutions, pairing residents with mentors 
experienced in the research process and motivated 
to produce high quality research may help achieve 
similar endpoints as it has with practitioners. 11 
Additionally, complementing this mentorship with 
individuals with expertise in targeted areas, such as 
biostatistics or administrative data queries, may 
further facilitate movement through the various 
stages of the research process. 

Second, while the vast majority of respondents 
stated a manuscript in a format suitable for 
publication was required for residency completion, 
very few residents published their work. 5 " 7 It has 
been previously remarked that residency research 
projects often fall short of the high standards set by 
peer-reviewed journals simply because projects are 
not given priority or designed for publication from 
the start. 15 If publishing resident projects is a goal of 
residency programs, then programs might consider 
allowing multiple residents to collaborate on a single 
project 6 , offering projects that span multiple years, 
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capitalizing on completed research with follow-up 
studies, or conducting a feasibility analysis prior to 
project initiation. 

Both groups identified difficulties ensuring project 
continuity and publication following residency 
completion as another barrier. Once again, 
assigning an experienced researcher from the 
institution to each resident project may help in 
developing a post residency communication plan to 
ensure that projects do not languish. Additionally, 
setting clear expectations at the start of the 
residency about submitting and publishing the study 
results may help alleviate this barrier. Finally, a 
newly created journal dedicated to publishing 
pharmacy resident (past and present) projects 
offers another option for submission of these 
projects which may improve publication rates. 17 

Our study has limitations. We had a low response 
rate from program directors despite repeated 
reminders. While we still feel that the responses we 
received provided valuable insight in barriers to 
residency research training there is the possibility 
that these results are not reflective of the entire 
population of residents and program directors. 
Second, the survey responses were anonymous 
and thus we do not know specifically which 
programs responded which could have affected the 
results, as residency programs with a larger 
infrastructure may be more likely to have current 
contact information for past residents. This bias may 
be supported by the fact that 15.5% of residents 
stated that their residency project was either 
published or in press. This percentage is higher 
than that previously reported for WSC research 
projects of less than 10% 6,7 , although consistent 
with a study of the SERC. 5 Our inquiry about 
whether programs had a "formal research process" 
was not specific thus may have been interpreted 
differently among respondents. 



CONCLUSIONS 

In conclusion, it appears both residency program 
directors and residents agree that time constraints 
and low study quality are major factors limiting 
publication of resident research projects. However, 
program directors also felt that residents lacked the 
motivation to work through the publication process 
while residents felt poor mentorship and a lack of 
program support were larger factors. If programs 
wish to improve the publication rate of residency 
research projects, results of this study may provide 
some opportunities for improvement including 
formal research training that includes information on 
biostatistics, a plan for continued communication 
between resident and mentor upon residency 
completion, and an increase in the time allocated for 
research. Additionally, if developing pharmacy 
researchers is a goal of our profession, then 
perhaps strengthening the language in the 
accreditation standards to specifically say 
"research" may push programs to develop stronger 
original research projects. 
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